To this end, high-probability guarantees have been considered in the literature [35, 64, 20, 32, 22]. These results allow to control the risk associated with the worst-case tail events as theyspecify howmanyiterations would be sufficient toensureG(xk,yk) issufficiently small foranygivenfailure probability q (0,1).

Then momentum acceleration is also introduced into our dual accelerating update step.

Then momentum acceleration is also introduced into our dual accelerating update step.

By replacing the inner optimizer in RAPP we rediscover the family of Lookahead algorithms for which we establish convergence in cohypomonotone problems even when the base optimizer is taken to be gradient descent ascent.

It also employs a new shuffling estimator for the "hyper-gradient",

Neural Information Processing Systems http://nips.cc/